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1.  Introduction 


The  US  Army  Research  Laboratory’s  (ARL)  Acoustic  and  Electromagnetic 
Sensing  Branch  is  developing  sensing  technologies  that  use  acoustic  and  seismic 
methods  to  detect  sound  and  vibrations  on  and  off  the  battlefield.  Many  of  these 
methods  use  traditional  sensors  such  as  microphones,  lasers,  geophones,  and 
accelerometers.  Each  of  the  sensors  have  their  own  limitations  and  advantages  as 
they  perform  differently  in  terms  of  range,  accuracy,  speed,  concealment,  and  so 
on.  ARL  is  currently  investigating  using  video  cameras  to  measure  vibrations  of 
objects.  ARL  and  Booz  Allen  Hamilton  (BAH)  collaborated  on  a  field  test  in  June 
2016  to  evaluate  the  feasibility  of  using  video  cameras  to  detect  vibrations  excited 
by  low-frequency  soundwaves.  Although  video  cameras  will  not  be  as  sensitive  as 
other  traditional  sensors,  cameras  are  ubiquitous  in  society  and  can  potentially  be 
used  to  provide  additional  information  to  the  Soldier. 

2.  Motivation  and  Vision  for  Military  Applications 

Video  cameras  have  several  advantages  over  other  sound  detection  devices.  These 
devices  tend  to  be  inexpensive  compared  to  other  technology,  such  as  laser  sensors 
and  radar  detection  systems.  A  recent  study,  led  by  a  collaboration  between 
scientists  and  researchers  of  the  Massachusetts  Institute  of  Technology,  Microsoft, 
and  Adobe,  utilized  high-speed  video  cameras  that  sampled  between  2  to  20  kHz 
to  monitor  human  conversation.1 

The  goal  of  this  experiment  was  to  reconstruct  human  speech  by  extracting  audio 
from  video  data.  Video  cameras  recorded  the  vibrational  behavior  of  a  variety  of 
objects  as  they  were  excited,  each  with  different  material  characteristics.  A  potted 
plant,  bag  of  chips,  box  of  tissues,  and  a  glass  of  water  were  the  objects  of 
concentration  in  this  experiment.  Conducting  separate  tests,  each  object  was  placed 
at  a  distance  ranging  approximately  0.5  to  2  m  away  from  the  source.  During  each 
test  run,  live-speech  conversation  was  amplified,  which  excited  the  specific  object 
of  interest.  The  vibrations  of  the  objects  were  recorded  using  the  high-speed 
cameras.  Image  processing  algorithms  and  acoustic  techniques  were  used  to  play 
back  the  original  sound  source.  This  research  demonstrated  that  video  cameras  can 
be  used  for  more  than  just  capturing  visual  images  for  the  Army.  In  summary,  video 
cameras  have  the  capability  to  enhance  surveillance  mechanisms  and  provide  a 
means  for  detecting  sound. 
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3.  Methods,  Assumptions,  and  Procedures 


3.1  Location 

In  June  2016,  ARL  conducted  a  field  test  with  BAH  at  ARL  Building  108  to  test 
the  feasibility  of  using  video  cameras  to  detect  low-frequency  tones  within  a 
building.  Building  108  is  a  single-story  building  that  contains  a  large  room.  It  is  a 
wood-framed  building,  covered  with  a  vinyl  exterior.  Within  the  main  room  of  the 
building,  desks  and  computers  are  stationed  to  support  ongoing  research.  A 
ServoDrive  speaker  located  in  Building  108  was  used  to  generate  tones.  The 
windows  of  the  building  are  made  of  glass  and  are  enclosed  by  wooden  frames.  The 
building  was  chosen  for  its  convenience  in  location,  size,  and  material 
characteristics.  Figures  1-4  show  images  of  Building  108.  In  Fig.  1,  an  image  is 
provided  of  the  back  side  of  Building  108.  The  windows  are  evenly  spaced  out 
along  the  building.  Ground-truth  sensors  and  video  cameras  were  positioned  in  the 
grass  space  illustrated  in  the  figure.  Figure  2  shows  the  roadside  view  of  Building 
108.  The  entrance  to  the  building  is  shown  in  this  image.  The  building’s  size  in  its 
entirety  is  shown  in  the  side  view  provided  in  Fig.  3.  Figure  4  shows  the  spacious 
room  inside  of  Building  108  used  for  testing  the  ServoDrive  speaker. 


Fig.  1  Building  108  (back  view) 
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Fig.  2  Building  108  (roadside  view) 
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Fig.  4  Inside  Building  108  (test  room),  location  of  ServoDrive  speaker 

3.2  Test  Summary 

Imagery  of  a  door  and  a  window  were  collected  using  high-speed  video  cameras. 
Supporting  ground-truth  data  for  the  excitation  were  collected  using  acoustic 
microphones,  a  laser  Doppler  vibrometer  (LDV),  and  geophones.  The  speaker 
generated  tones  at  19,  28,  and  37  Hz.  The  video  cameras  used  were  a  Point  Grey 
Camera,  a  RED  Scarlet  Dragon  Camera,  and  a  Casio  Camera  that  sampled  at  200, 
240,  and  1,200  fjps,  respectively.  Data  files  measured  by  these  cameras  can  be 
accessed  in  ARL’s  Automated  Online  Data  Repository  (AODR)  database.2,3 

4.  Truth  Sensors  and  Supporting  Technologies  Specifications 

This  experiment  utilized  high-speed  video  cameras  to  record  the  local  vibrations  of 
a  window  and  door  of  a  building  as  they  were  excited  by  low-level  frequency  waves 
generated  by  the  ServoDrive  speaker.  The  following  cameras  were  used  to  record 
data:  RED  Camera,  Casio  Camera,  and  Point  Grey  Camera.  Table  1  characterizes 
the  3  video  cameras  and  the  supporting  sensing  devices  used  for  collecting  ground 
truth  data.  The  table  lists  the  sensing  capability  of  each  device,  gives  the  model 
specifications,  and  provides  an  image  of  the  model  of  the  device  used  during  the 
experiment. 
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Table  1  Cameras  and  truth  sensors 


Sensor 

Casio  Camera 
Exlim  Pro  (Ex-FI) 


RED  Scarlet  Dragon  Camera 


Point  Grey  Video  Camera 


Briiel  &  Kjaer  Microphone 


Briiel  &  Kjaer  Foam 
Microphone 


ServoDrive  Speaker 


Specifications  Image 

512*384  @  300  fps,  432*192  @ 

600  fps,  336*96  @  1200  fps 
6.0  Mega  Pixels 
2.8”  Wide  &  Bright  LCD 

6k  @  12fps,  5k  @48fps,  4k  @  60 
fps,  3k  @  80  fps,  2k  @  120  fps 
Rolling  Shutter 
S/N  Ratio:  80dB 
Dynamic  Range:  16.5+  Stops 

200  fps 


Type  2669  No.  1856889  23/07 
(4193  -  20  Gain  dB) 


Type  2669  No.  2344712  (4192  -  20 
Gain  dB) 


ITC  License 

CB- 1 00 1  - 1 36-ETC2/03 

Glenview,  Illinois 
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Sensor  Specifications 

National  Instruments  (NI)  cRIO  -  9025 

CompactRIO  Real-Time 

Controller 


Polytech  Laser  Doppler 
Vibrometer 


Polytech  PDV-100 
Frequency  range:  0  Hz-22  kHz 
Range:  0.1-30  m 


Geospace  Geophone 


Houston,  Texas 
Patent  4594695 


Image 


Table  2  lists  supporting  technology  and  power  supplies  used  to  support  the  camera 
and  truth  sensors.  These  devices  are  listed  by  their  name,  model  specifications,  and 
a  photo  of  the  model  of  the  device  used  during  the  experiment.  The  Dual 
Microphone  Supply  supported  the  2  B&K  microphones  specified  in  Table  1.  The 
Crown-Tech  600  was  used  to  drive  the  ServoDrive  speaker.  A  Dell  laptop  owned 
by  BAH  was  used  during  the  experiment  for  processing  and  moving  data  files. 
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Table  2  Supporting  devices  and  technology 


Sensor  Specifications  Image 

Dual  Microphone  Supply  Type  5935L 


Crown  Micro-Tech  600 


Data  acquisition  laptop 


Lithium  ion  (Li-ION)  24  V,  7.5  Ah  at  2  A,  BB-2590/U 

Sealed:  Rechargeable 

battery 


Synthesizer/Function 
Generator  Hewlett  Packard 


5.  The  Experiment 

During  this  experiment,  truth  sensors  were  positioned  at  specific  distances  away 
from  the  ARL  Building  108.  An  LDY  sensor  was  located  approximately  25  ft  away 
from  the  building,  while  the  geophone  and  2  microphones  were  each  20  ft  away 
from  the  building.  The  Casio,  RED,  and  Point  Grey  cameras  were  each  stationed 
approximately  30,  30,  and  15  ft,  respectively,  from  the  building.  To  note,  the  Point 
Grey  Camera  was  moved  to  a  greater  standoff  distance  of  130  ft  later  in  the 
experiment.  The  cameras  and  supporting  sensors  recorded  data  as  the  speaker 
generated  tones  with  frequencies  at  19,  28,  and  37  Hz.  These  tones  were  purposely 
restricted  to  low  frequencies. 


Approved  for  public  release;  distribution  is  unlimited. 

7 


The  cameras  were  focused  on  either  the  door  or  a  window.  One  set  of  tests  used  the 
cameras  and  truth  sensors  to  record  the  excitation  of  a  door  on  the  building,  while 
the  other  test  focused  on  the  excitation  of  a  window.  The  door  test  began  with  taking 
a  baseline  measurement  of  the  door.  This  measurement  was  conducted  and  recorded 
4  consecutive  times.  The  files  for  these  data  runs  can  be  found  under  the  “Data 
Collection  Documented”  section  of  this  report  in  Tables  3-7.  Video  data  were 
collected  as  the  speaker  emitted  frequencies  at  19,  28,  and  37  Hz.  Approximately 
30  s  of  data  were  taken  during  4  runs.  After  completing  the  door  test,  the  window 
test  was  conducted. 

Images  of  the  experimental  setup  have  been  provided  in  Figs.  5-8.  Figure  5  is  an 
image  of  both  the  door  and  window  that  were  excited  by  the  ServoDrive  speaker. 
A  piece  of  retroreflective  tape  was  placed  on  both  objects  to  provide  a  reflection 
for  the  LDV.  A  wide-range  image  of  the  experimental  setup  is  shown  in  Fig.  6.  This 
image  displays  the  orientation  setup  of  the  Casio,  RED,  and  Point  cameras,  as  well 
as  other  supporting  sensors.  Figure  7  shows  the  setup  of  the  Casio  camera,  RED 
camera,  and  LDV  in  relation  to  each  other  as  they  are  facing  the  backside  of  the 
building.  Figure  8  shows  a  close-up  of  the  LDV.  To  note,  during  the  experiment, 
the  LDV  at  one  point  overheated.  For  the  duration  of  the  experiment,  an  umbrella 
was  used  to  cover  it  to  prevent  overheating. 


Fig.  5  Zoomed-in  image  of  door  and  window  excited  by  ServoDrive  speaker 
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Fig.  6  Device  setup:  Casio  camera  (left),  LDV  camera  (center),  RED  camera  (right) 


Fig.  7  View  of  cameras  and  supporting  technology 


Approved  for  public  release;  distribution  is  unlimited. 

9 


Fig.  8  Umbrella  compensates  for  LDV  overheating  during  test  run 

6.  Signal  and  Image  Processing:  Data  Analysis 

ARL’s  Acoustic  and  Electromagnetic  Sensing  Branch  uses  2  techniques  to  process 
video  data.  The  first  method  requires  taking  a  discrete  Fourier  transform  (DFT)  of 
several  images  that  are  taken  across  the  time  domain.  These  images  are  first  stacked 
on  top  of  each  other  and  then  a  DFT  is  performed.  Finally,  the  results  of  the  DFTs 
associated  with  a  small  region  of  interest  are  noncoherently  averaged.  For  the  other 
method,  the  consecutive  images  are  correlated  to  estimate  a  pixel  shift,  then  a  DFT 
of  the  results  is  calculated.  In  this  report,  the  second  method  was  used  to  process 
selected  data.  Video  data  were  collected  from  the  Casio,  RED,  and  Point  Grey 
cameras,  as  well  as  recorded  using  the  LDV,  geophone,  CompactRIO  (cRIO) 
systems,  and  the  microphones.  These  devices  served  as  ground-truth  data  devices 
to  verify  the  main  data  collects,  the  Casio  and  RED  data  files.  Video  data  were 
stored  in  .MOV,  .AVI,  and  .R3D  video  files,  and  the  ground-truth  data  were  stored 
in  .BIN  files.  Imaging  data  were  collected  during  the  day  in  a  relatively  quiet 
environment.  Approximately  100  GB  of  data  were  collected.  Tables  3-7  list  the 
data  files  collected  during  the  experiment  by  the  various  sensors. 

7.  Data  Collection  Documented 


7.1  Collection  Process 

The  field  test  data  collection  process  started  at  approximately  2:30  PM  and 
concluded  at  approximately  6:00  PM.  The  following  data  tables  list  the  data  files 
that  were  recorded  by  the  cameras  and  ground-truth  sensors  during  testing.  Table  3 
documents  video  files  recorded  by  the  Casio  Camera.  These  files  were  stored  in 
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.MOV  format.  Approximately  892  MB  of  data  were  stored  by  the  camera.  These 
data  files,  on  average,  took  up  about  53  MB  of  storage  for  each  file.  In  each  of  these 
data  collects,  the  Casio  camera  recorded  either  the  door  or  window.  The  table 
specifies  the  object  of  focus  during  each  test  run.  The  table  also  includes  which 
frequency  the  object  was  excited  at  and  the  time  the  test  run  occurred.  The  last 
column  of  the  table  states  how  large  each  data  file  is.  There  is  a  division  between  2 
sets  of  data  taken  by  the  camera  in  the  table.  During  round  1  of  the  data  collection, 
the  Casio  camera  was  stationed  30  ft  away  from  the  building.  During  round  2,  the 
camera  was  moved  to  a  standoff  distance  of  approximately  130  ft  away  from  the 
building.  Approximately  100  GB  of  data  were  collected  during  the  test. 


Table  3  File  descriptions  for  Casio  camera  data 


File  Name 

Object  Excited 

Frequency  (Hz) 

Time  Stamp 

File  Size 

ROUND  1 

ROUND  1 

ROUND  1 

ROUND  1 

ROUND  1 

(30  FT) 

(30  FT) 

(30  FT) 

(30  FT) 

(30  FT) 

CIMG0006.JPEG 

Door 

1.63  MB 

CIMG0006.mov 

Window 

74.1  MB 

CIMG0007.JPEG 

Door 

3:52 

1.61  MB 

CIMG0007.mov 

Window 

85.7  MB 

CIMG0008.mov 

Window 

37 

58.6  MB 

CIMG0009.mov 

Window 

37 

68.6  MB 

CIMGOOlO.mov 

Window 

37 

4:28 

52.1  MB 

CIMGOOll.mov 

Window 

28 

4:38 

63.8  MB 

CIMG0012.mov 

Window 

28 

4:41 

42.8  MB 

ROUND  2 

ROUND  2 

ROUND  2 

ROUND  2 

ROUND  2 

(130  FT) 

(130  FT) 

(130  FT) 

(130  FT) 

(130  FT) 

CIMG0002.mov 

Window 

19 

4:50 

69.6  MB 

CIMG0003.mov 

Window 

19 

4:53 

58.9  MB 

CIMG0004.mov 

Window 

Baseline 

4:57 

58.1  MB 

CIMG0005.mov 

Door 

Baseline 

5:25 

54.5  MB 

CIMG0008.mov 

Door 

19 

5:35 

35.8  MB 

CIMG0009.mov 

Door 

37 

5:39 

61.8  MB 

CIMG000 1 0.mov 

Door 

37 

5:40 

52.8  MB 

CIMGOOll.mov 

Door 

Baseline 

51.5  MB 

Video  files  recorded  by  the  RED  Camera  are  listed  in  Table  4.  These  files  were 
stored  in  .RDC  format.  Approximately  30  GB  of  data  were  stored  by  the  camera. 
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These  data  files,  on  average,  took  up  about  1.7  GB  of  storage  for  each  file.  In  each 
of  these  data  collects,  the  RED  camera  recorded  either  the  door  or  window.  The 
table  specifies  the  object  of  focus  during  each  test  run.  The  table  also  includes 
which  frequency  the  object  was  excited  at  and  the  time  the  test  run  occurred.  The 
last  column  of  the  table  states  how  large  the  data  file  is.  The  video  files  were  taken 
30  ft  away  from  the  building. 


Table  4  File  descriptions  of  RED  camera  data 


File  Name 

Object  Excited 

Frequency  (Hz) 

Time  Stamp 

File  Size 

A004_C03 5_060 157 .  RDC 

Door 

1.40  GB 

A004_C03 6_060 1 QS .  RDC 

Door 

717  MB 

A004_C03  8_060 1 07.  RDC 

Window 

1.00  GB 

A004_C03 9_060 1 F4.  RDC 

Window 

503  MB 

A004_C042_060 1 WO.  RDC 

Window 

Baseline 

2.46  GB 

A004_C043_0601ZZ.RDC 

Window 

1.77  GB 

A004_C044_060 1 4R.  RDC 

Window 

2.12  GB 

A004_C045_060 1 IB .  RDC 

Window 

404  MB 

A004_C046_060 1 MS .  RDC 

Window 

237  MB 

A004_C047_060 1 HO.  RDC 

Window 

37 

4:28 

2.10  GB 

A004_C048_06018Q.RDC 

Window 

28 

4:38 

2.46  GB 

A004_C049_060 1  SQ.RDC 

Window 

28 

4:41 

2.26  GB 

A004_C050_0601 8 Y.RDC 

Window 

19 

4:50 

2.59  GB 

A004_C05 1_0601XC.RDC 

Window 

19 

4:53 

2.15  GB 

A004_C052_060 1  OW.RDC 

Window 

Baseline 

4:57 

2.10  GB 

A004_C05 3_060 1 87 .  RDC 

Door 

Baseline 

5:25 

2.11  GB 

A004_C054_060 1 53  .RDC 

Door 

19 

2.11  GB 

A004_C05 5_060 1 H5 .  RDC 

Door 

19 

5:35 

2.30  GB 

A004_C05 6_060 1 ME  .RDC 

Door 

37 

5:39 

2.27  GB 

A004_C057_060 1 RY.RDC 

Door 

37 

5:40 

2.09  GB 

A004_C05  8_060 1 9D.  RDC 

Door 

Baseline 

2.07  GB 

A004_C05 9_060 1 Y3  .RDC 

Door 

975  MB 

A004_C060_060 1 XO.  RDC 

Door 

215  MB 

Video  files  recorded  by  the  Point  Grey  Camera  are  listed  in  Table  5.  These  files 
were  stored  in  .AVI  format.  Approximately  63  GB  of  data  were  stored  by  the 
camera.  These  data  files,  on  average,  took  up  about  1.7  GB  of  storage  for  each  file. 
In  each  of  these  data  collects,  the  Point  Grey  camera  recorded  imagery  of  either  the 
door  or  window.  The  table  specifies  the  object  of  focus  during  each  test  run.  The 


Approved  for  public  release;  distribution  is  unlimited. 


12 


table  also  includes  which  frequency  the  object  was  excited  at  and  the  time  the  data 
trial  occurred.  The  last  column  of  the  table  states  how  large  the  data  file  is.  There 
are  2  sets  of  data  taken  by  the  Point  Grey  camera  in  the  table.  During  round  1  of 
the  data  collect,  the  Point  Grey  camera  was  stationed  15  ft  away  from  the  building. 
During  round  2,  the  camera  was  moved  to  a  standoff  distance  of  30  ft  away  from 
the  building. 


Table  5  File  descriptions  of  Point  Grey  camera  data 


File  Name 

Object  Excited 

Frequency  (Hz) 

Time  Stamp 

File  Size 

ROUND  1 

ROUND  1 

ROUND  1 

ROUND  1 

ROUND  1 

(15  FEET) 

(15  FEET) 

(15  FEET) 

(15  FEET) 

(15  FEET) 

PG_WINDOW_B  ASELINE_C  1_0 

Window 

Baseline 

1.99  GB 

PG_WINDOW_B  ASELINE_C  1_1 

Window 

Baseline 

1.99  GB 

PG_WINDOW_B  ASELINE_C  1_2 

Window 

Baseline 

1.14  GB 

PG_WINDOW_BASELINE_C2_0 

Window 

Baseline 

1.99  GB 

PG_WINDOW_B  ASELINE_C2_  1 

Window 

Baseline 

1.99  GB 

PG_WINDOW_BASELINE_C2_2 

Window 

Baseline 

1.14  GB 

PG_WINDOW_B  ASELINE_C2_2_201 6- 

Window 

Baseline 

1.99  GB 

06-01-165710-0002 

PG_WINDO  W_SPE  AKER_  1 9Hz_C  1  _0 

Window 

19 

1.99  GB 

PG_WINDO  W_SPE  AKER_  1 9Hz_C  1  _0 

Window 

19 

1.99  GB 

PG_WINDO  W_SPE  AKER_  1 9Hz_C  1  _2 

Window 

19 

1.14  GB 

PG_WINDOW_SPEAKER_l  9Hz_C2_0 

Window 

19 

1.99  GB 

PG_WINDO  W_SPE  AKER_  1 9Hz_C2_  1 

Window 

19 

1.99  GB 

PG_WINDOW_SPEAKER_l  9Hz_C2_2 

Window 

19 

1.14  GB 

PG_WINDOW_SPEAKER_28Hz_C  1_0 

Window 

28 

1.99  GB 

PG_WINDOW_SPEAKER_28Hz_C  1_1 

Window 

28 

1.99  GB 

PG_WINDOW_SPEAKER_28Hz_C  1_2 

Window 

28 

1.14  GB 

PG_WINDOW_SPEAKER_28Hz_C2_0 

Window 

28 

1.99  GB 

PG_WINDOW_SPEAKER_28Hz_C2_l 

Window 

28 

1.99  GB 

PG_WINDOW_SPEAKER_28Hz_C2_2 

Window 

28 

1.14  GB 

PG_WINDOW_SPEAKER_37Hz_C  1_0 

Window 

37 

1.99  GB 

PG_WINDOW_SPEAKER_37Hz_C  1_1 

Window 

37 

1.99  GB 

PG_WINDOW_SPEAKER_37Hz_C  1_2 

Window 

37 

1.14  GB 

PG_WINDOW_SPEAKER_37Hz_C2_0 

Window 

37 

1.99  GB 

PG_WINDOW_SPEAKER_37Hz_C2_l 

Window 

37 

1.99  GB 

PG_WINDOW_SPEAKER_37Hz_C2_2 

Window 

37 

1.14  GB 
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File  Name 

Object  Excited 

Frequency  (Hz) 

Time  Stamp 

File  Size 

ROUND  1 

ROUND  1 

ROUND  1 

ROUND  1 

ROUND  1 

(15  FEET) 

(15  FEET) 

(15  FEET) 

(15  FEET) 

(15  FEET) 

PG_WINDOW_SPEAKER_37Hz_C3_0 

Window 

37 

1.99  GB 

PG_WINDOW_SPEAKER_37Hz_C3_l 

Window 

37 

1.99  GB 

PG_WINDOW_SPEAKER_37Hz_C3_2 

Window 

37 

1.14  GB 

PG_WINDOW_SPEAKER_37Hz_C4_0 

Window 

37 

1.99  GB 

PG_WINDOW_SPEAKER_37Hz_C4_l 

Window 

37 

1.99  GB 

PG_WINDOW_SPEAKER_37Hz_C4_2 

Window 

37 

1.14  GB 

ROUND  2 

ROUND  2 

ROUND  2 

ROUND  2 

ROUND  2 

(30  FEET) 

(30  FEET) 

(30  FEET) 

(30  FEET) 

(30  FEET) 

PG_WINDOW_37Hz_FAR_C  1_20 1 6-06- 

Window 

37 

1.99  GB 

01-175 822-0000.  avi 

PG_WINDOW_37Hz_FAR_C  1_20 1 6-06- 

Window 

37 

1.99  GB 

01-175822-0001.avi 

PG_WINDOW_37Hz_FAR_C  1_20 1 6-06- 

Window 

37 

1.14  GB 

01-17 5 822-0002.  avi 

PG_WINDOW_37Hz_FAR_C2_20 1 6-06- 

Window 

37 

1.99  GB 

01-1 80022-0000.  avi 

PG_WINDOW_37Hz_FAR_C2_20 1 6-06- 

Window 

37 

1.99  GB 

01-1 80022-000 1 .  avi 

PG_WINDOW_37Hz_FAR_C2_20 1 6-06- 

Window 

37 

1.14  GB 

01-1 80022-0002.  avi 

Table  6  lists  the  7  connection  ports  that  were  used  by  the  cRIO  data  acquisition 
device.  Each  of  the  7  channels  was  connected  to  one  of  the  ground-truth  sensors. 
Channel  1  was  connected  to  the  LDV;  Channel  2  was  connected  to  the  foam  ball 
microphone;  Channel  4  was  connected  to  the  2-3  port  of  the  geophone;  Channel  5 
was  connected  to  the  2-4  port  of  the  geophone;  Channel  6  was  connected  to  the  2- 
5  port  of  the  geophone;  and  Channel  7  was  connected  to  the  ball  microphone 
without  foam.  Channel  3  of  the  cRIO  was  not  connected  to  any  input  device. 
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Table  6  cRIO  channel  connection  descriptions 


Channel 

Sensor 

1 

Laser  Doppler  vibrometer 

2 

B&K  microphone  with  foam  ball  4192-20  gain  dB 

3 

Disconnected 

4 

2-3  Geophone  port 

5 

2-4  Geophone  port 

6 

2-5  Geophone  port 

7 

B&K  microphone  ucOl  1  adapter  bmk  4193  -  20  gain  dB 

Table  7  lists  the  data  files  recorded  by  the  cRIO  data  acquisition  system.  These  files 
were  stored  in  .BIN  format.  Approximately  88  MB  of  data  were  stored  by  the  cRIO 
device.  These  data  files  took  up  about  6  MB  of  storage  on  average  for  each  file.  In 
each  of  these  data  collects,  the  cRIO  device  recorded  either  the  door  or  window. 
The  table  specifies  the  object  of  focus.  The  table  also  includes  which  frequency  the 
specified  object  of  interest  was  excited  at  and  the  time  the  data  trial  occurred.  The 
last  column  of  the  table  states  how  large  the  data  file  is. 


Table  7  File  descriptions  of  cRIO  data 


File  Name 

Object  Excited 

Frequency  (Hz) 

Time  Stamp  File  Size 

AD_DOOR_BASELINE_C  1  .bin 

Door 

Baseline 

6.48  MB 

AD_DOOR_SPEAKER_l  9Hz_C  1  .bin 

Door 

19 

2.47  MB 

AD_DOOR_SPEAKER_l  9Hz_C2.bin 

Door 

19 

4.57  MB 

AD_DOOR_SPEAKER_37Hz_C  1  .bin 

Door 

37 

6.10  MB 

AD_DOOR_SPEAKER_37Hz_C2.bin 

Door 

37 

4.00  MB 

AD_WINDOW_BASELINE_C  1  .bin 

Window 

Baseline 

6.67  MB 

AD_WINDOW_BASELINE_C2.bin 

Window 

Baseline 

6.10  MB 

AD_WINDOW_SPEAKER_l  9Hz_C  1  .bin 

Window 

19 

7.24  MB 

AD_WINDOW_SPEAKER_l  9Hz_C2.bin 

Window 

19 

6.10  MB 

AD_WINDOW_SPEAKER_28Hz_C  1  .bin 

Window 

28 

7.24  MB 

AD_WINDOW_SPEAKER_28Hz_C2.bin 

Window 

28 

6.29  MB 

AD_WINDOW_SPEAKER_37Hz_C  1  .bin 

Window 

37 

6.29  MB 

AD_WINDOW_SPEAKER_37Hz_C2.bin 

Window 

37 

6.48  MB 

AD_WINDOW_SPEAKER_37Hz_C3  .bin 

Window 

37 

7.81  MB 

AD_WINDOW_SPEAKER_37Hz_C4.bin 

Window 

37 

4.18  MB 
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7.2  Sample  Example  of  Processing  Data  Files 


The  following  figures  display  some  of  the  images  and  graphs  that  represent  data 
generated  from  ARL’s  image  and  signal  processing  algorithms.  Figure  9  shows  2 
images  taken  by  the  RED  and  Casio  cameras  of  the  reflective  tape  that  was  used  on 
the  door  to  obtain  adequate  measurements  from  the  LDY. 


RED  Casio 


Fig.  9  Door  experiment:  speaker  at  19  Hz,  visual  image  through  camera  lens:  RED  (left), 
Casio  (right) 

Figure  10  shows  the  spectrum  of  the  processed  data  versus  the  time  of  the  door 
being  excited  by  a  19-Hz  tone.  The  solid  yellow  lines  in  the  2  graphs  verify  that  a 
19-Hz  signal  was  detected  by  both  the  RED  and  Casio  cameras.  This  yellow  line 
clearly  stands  out  in  both  graphs. 


19  Hz 


5  10  15  20  25  30  2  4  6  &  10  12  14  16  tfl 

tim*  time  (sec) 
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*5 


-30 


'35 


•45 


Fig.  10  Door  experiment:  speaker  at  19  Hz,  frequency  vs.  time:  RED  (left),  Casio  (right) 

Figures  1 1  through  15  display  the  spectrum  versus  time  measurements  for  the  LDV, 
microphones,  and  geophones.  These  graphs  verify  that  the  speaker  generated  tones 
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at  19  Hz.  These  figures  provide  ground  truth  data  for  the  RED  and  Casio  cameras, 
confirming  that  the  data  results  taken  from  the  cameras’  readings  are  accurate.  In 
Fig.  11,  the  LDV  and  the  microphone  data  show  a  clear  indication  of  detection  of 
19  Hz  tone,  while  the  results  for  the  2-3  geophone  connection  data  are  a  little  less 
evident.  The  harmonics  19,  38,  57,  76,  and  95  are  clearly  evident  in  the  LDV  graph. 
The  solid  reddish-brown  line  in  the  microphone  data  graph  shows  a  clear  indication 
of  the  detection  of  the  19-Hz  tone. 


Fig.  11  Door  experiment:  speaker  at  19  Hz,  frequency  vs.  time:  LDV  (left),  foam  mic 
(middle),  2-3  geophone  connect  (right) 


Figure  12  displays  2  data  graphs  of  the  spectrum  versus  time  of  the  LDV  data.  The 
graph  on  the  right  illustrates  a  zoomed- in  view  of  the  data  plotted  in  the  left  graph. 
The  right-hand  graph  clearly  shows  the  fundamental  frequency  at  19  Hz  and  its 
harmonics. 


Spectrum  of  Laser  Doppler  Vibrometer 

Spectrogram  Time  21:31:23.1043 


Laser  Doppler  Vibrometer  Zoomed-ln 


Fig.  12  Door  experiment:  speaker  at  19  Hz,  frequency  vs.  time:  LDV  data  (left),  LDV  data 
zoomed-in  (right) 

Figure  13  displays  the  spectrum  of  2  data  graphs  that  represent  data  collected  by 
the  B&K  microphone  with  a  foam  wind  screen.  The  graph  on  the  right  illustrates  a 
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zoomed- in  view  of  the  data  plotted  in  the  left  graph.  The  right-hand  graph  clearly 
shows  the  fundamental  frequency  at  19  Hz. 


B&K  Microphone  with  foam  ball  wind  screen 


Foam  Ball  B&K  Microphone  Zoomed-ln 


Spectrogram  Time  21:31:23.1043 
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Fig.  13  Door  experiment:  speaker  at  19  Hz,  frequency  vs.  time:  foam  mic  data  (left),  foam 
mic  data  zoomed-in  (right) 

Figure  14  displays  2  data  graphs  that  represent  data  collected  by  the  geophone 
connection  labeled  2-3.  The  graph  on  the  right  illustrates  a  zoomed-in  view  of  the 
data  plotted  in  the  left  graph.  The  detection  of  the  19-Hz  signal  in  data  graph  on  the 
right  is  less  obvious  compared  to  data  collected  by  the  cameras,  microphones,  and 
the  LDV. 


2-3  Geophone  2-3  Geophone  Zoomed-ln 


Fig.  14  Door  experiment:  speaker  at  19  Hz,  frequency  vs.  time:  2-3  geophone  connect  data 
(left),  2-3  geophone  connect  data  zoomed-in  (right) 

Figure  15  displays  the  frequency  versus  time  graphs  for  the  2  geophone  channels, 
2-4  and  2-5,  as  well  as  the  microphone  without  a  wind  screen.  The  geophone  and 
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Frequency 


microphone  results  both  show  a  tone  at  of  19  Hz.  The  geophone  results  have  lower 
SNRs  compared  to  the  other  sensors  used  in  this  experiment. 


2-4  Geophone 


2-5  Geophone 


B&K  Microphone 


Fig.  15  Door  experiment:  speaker  at  19  Hz,  frequency  vs.  time:  2-4  geophone  (left),  2-5 
geophone  connect  (center),  standard  mic  (right) 

The  data  were  affected  by  several  factors  during  the  experiment.  Specifically,  wind 
affected  imagery  by  blowing  the  leaves  and  trees  in  the  imagery.  In  addition,  noise 
from  ongoing  traffic  coming  from  Powder  Mill  Road  near  ARL’s  facility  may  have 
contributed  to  additional  noise  noticeable  in  the  processed  imagery.  In  this 
experiment,  there  were  a  few  time  delays  in  the  start  of  the  devices.  The  cRIO 
device  had  a  time  delay.  In  addition,  sometimes  the  devices  did  not  start  in  sync 
with  the  other  devices,  or  they  would  shut  off  during  the  experiment. 

8.  Conclusions  and  Considerations 


ARL  and  BAH  conducted  a  field  test  that  was  designed  to  assess  how  high-speed 
video  cameras  can  be  used  to  detect  sound.  The  team  measured  video  camera  data 
from  a  RED  Scarlet  Dragon  Camera,  a  Casio  Camera,  and  a  Point  Grey  Camera 
while  a  building  was  excited  by  a  low-frequency  ServoDrive  speaker  inside  of 
ARL’s  Building  108  facility.  The  field  test  and  the  data  collected  are  documented 
in  this  report.  The  team  developed  image  processing  algorithms  to  analyze  the 
results  and  successfully  verified  that  the  cameras  and  other  devices  correctly 
estimated  the  frequency  of  the  tone  generated  at  each  trial. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


AODR 

Automated  Online  Data  Repository 

ARL 

US  Army  Research  Laboratory 

BAH 

Booz  Allen  Hamilton 

cRIO 

CompactRIO 

DFT 

discrete  Fourier  transform 

LDV 

laser  Doppler  vibrometer 

SNR 

signal-to-noise  ratio 
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